\section{Design of Experiments}
\label{sec:experiments}

Since the assignment asks us to perform two well defined tasks, our experiments
revolve around them as well. In this section we will demonstrate the various experiments
that we tried. Section \ref{sec:results} documents the results of these experiments for 
the two tasks.

\subsection{Task 1: Predicting Links}
The first task is to predict the existence of a link between two terrorist nodes. As mentioned
in Section \ref{sec:dataset}, there are $851$ such known links. Also as mentioned in the same
section, we do not distinguish between type of links while constructing the adjacency
matrix for this task. We consider five different feature combinations
and measure the performance of our model using cross validation. 
All the experiments are part of 
learning a binary classifier using logistic regression
which has positive class depicting ``link exists" and negative class depicting ``link does not
exist."

\subsubsection{Concatenation of Features}
In our first experiment, we concatenate the attributes of each of the nodes 
in the potential link and learn from the aggregated information. From these 
features, we train our models to learn the properties of the two node
that increases its probability to form a link. 

\subsubsection{Pointwise Multiplication of Features}
We also experiment with pointwise multiplication of feature vectors of two
nodes for each potential link. From these features, our model can learn the
interactions between nodes which can result in a link.

\subsubsection{Features from Network Structure}
In addition to using pointwise multiplication on the features of the
individual nodes, we also try this method on the top $k$ features obtained
for each node from the low-rank matrix decomposition on the adjacency
matrix, from which we can learn the properties of network structure between
the nodes.

\subsection{Task 2: Predicting Type of Links}
Our second task is to predict the type of a known link between two nodes.
We consider binary classification with missing values since there is no type information 
about links that are not present among terrorists. This makes this task significantly
different from task 1, in which the type of the link was not important and hence, 
which was a binary classification problem with two classes: present and absent links.
We handle the multiple classes in this task as discussed in Section \ref{sec:preprocess}.
As in task 1, all experiments for task 2 are part of learning a binary classifier
using logistic regression which has positive class depicting ``organizational link"
and negative class as ``non-organizational link."

\subsubsection{Feature Combinations}
Similar to task 1, we experiment with feature concatenation and pointwise
feature multiplication of the given $612$ features as well as featured
obtained from the network structure by singular value decomposition of the
adjacency matrix.

\subsection{Cross Validation}
For every experiment we evaluate our model using a $10$ fold cross validation.
We choose AUC as the measure of performance in task 1, since the positive to negative class ratio is highly unbalanced for this task. 
The positive and negative classes in task 2 are well-balanced, and
hence, we choose 0/1 accuracy in addition to AUC as out measure of
performance in task 2.

\subsection{Selection of Rank}
We also experiment with rank of matrix factorization to get the best performance.
We perform these experiments for both the tasks since the network structure
is different for each task, hence we expect the number of network features to
be useful to be different for each task.
